Supplementary Material Random Feature Expansions for Deep Gaussian Processes

Random Feature Expansions for Deep Gaussian Processes Kurt Cutajar 1 Edwin V. Bonilla 2 Pietro Michiardi 1 Maurizio Filippone 1 A. Additional Experiments Using the experimental set-up described in Section 4, Figure 1 demonstrates how the competing models perform with regards to the RMSE (or error rate) and MNLL metric when two hidden layers are incorporated into the competing models. The results follow a similar progression to those reported in Figure 3 of the main paper. The DGP-ARC and DGP-RBF models both continue to perform well after introducing this additional layer. However, the results for the regularized DNN are notably inferior, and the degree of overfitting is also much greater. To this end, the MNLL obtained for the MNIST dataset is not shown in the plot as it was vastly inferior to the values obtained using the other methods. DGP-EP was also observed to have low scalability in this regard whereby it was not possible to obtain sensible results for the MNIST dataset using this configuration.